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Abstract 

We consider a single-hop switciied queueing network. Amongst a 
plethora of applications, these networks have been used to model wireless 
networks and input queued switches. The MaxWeight scheduling policies 
have proved popular, chiefly, because they are throughput optimal and do 
not require explicit estimation of arrival rates. 

In this article, we prove the same throughput optimality property for 
a generalization of the MaxWeight policy called the MaxWeight- (ot, g) 
policy. In brief, given parameter q > and concave functions g = (gf- 
j £ J), the MaxWeight-(a, g) policy chooses a random schedule which 
solves 

maximize '^9j{sj)Qj 

over s £<S>, 

where Q — (Qj : j £ J7) is the vector of queue sizes and where, allowing 
for randomization, the maximization is taken over a set of admissible 
schedules < 5 >. These throughput optimal, myopic scheduling policies 
allow for scheduling choices similar to those found in bandwidth sharing 
networks - a further well studied model of Internet congestion. 

1 Introduction 

The MaxWeight scheduling policies were first introduced by Tassiulas and 
Ephremides [H] as a model of wireless communication. Their policy was 
applicable to the class of switched queueing networks, where there are 
constraints on which queues can be served simultaneously. Subsequently, 
as a model of Internet protocol routers, McKeown et al. 21! applied this 
paradigm to the example of input-queued switches and coined the term 
throughput optimal. 

Roughly stated, a queue scheduling policy is throughput optimal if it 
is stable for every arrival rate for which there exists a stabilizing policy. If 
the vector of arrival rates is known then throughput optimality is trivial: 
at each time, one can chose a random schedule whose average service rate 
dominates each arrival rate. However, in practice, explicit knowledge of 
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arrival rates is not achievable, particularly when rates may vary over time. 
A striking feature of the Max Weight is that it is throughput optimal and it 
is myopic - meaning that only current queue state information is required 
to choose a schedule. For this reason, the MaxWeight policy has proved 
popular. Accordingly, the pohcy has been generaUzed. Before proceeding 
with a formal analysis, let's informally discuss these generalizations; let's 
discuss Bandwidth Sharing Networks, a different but somewhat related 
model of Internet control; and let's discuss the MaxWeight- (a, p) policy 
from which this paper is titled. 

MaxWeight-/ 

Currently, MaxWeight and its generalizations can be expressed in terms 
of the following MaxWeight-/ policy. The MaxWeight-/ policy chooses a 
schedule at each time which optimizes 

maximize Sjfj {Qj ) 

over s G<<S> . 

Here Qj is the queue sizes for each queue j £ J , and Sj is the mean 
number of queue j jobs served under a random schedule in < 5 >. The 
choice of < <S > allows for randomization; nonetheless, randomization 
needn't be used as there always exists a deterministic solution to the 
above optimization. 

The MaxWeight policy introduced by Tassiulas and Ephremides [41] is 
the case where / is the identity function. A broad class of functions / are 
proven to be throughput optimal by Meyn [22]. Amongst these choices, 
the MaxWeight-Q policies have proved popular [Tl[36l[3ll[Ml[Ill[32]. Here, 
each /,■ is a power function: fj{sj) = s" . 

So, thus far, generalizations of the MaxWeight policy alter the de- 
pendence on queue sizes, Qj, with an appropriately chosen function, say 

fAQ,) = Q7- 

Bandwidth Sharing Networks 

Somewhat separate to this work on switch networks, researchers have con- 
sidered bandwidth sharing networks [5ni[Hl[31[Il[ISl[ni[ll[IZl[2S]. These 
continuous-time Markov processes form a flow level model of Internet file 
transfer. Here, using slightly unconventional notation, it is assumed that 
the bandwidth allocation achieved by a congestion control protocol solves 
a network utility optimization of the form 



maximize 




over s G<5> . (lb) 

Here Qj represents the queue size of class j jobs to be transfered across the 
network and Sj is the bandwidth allocated to class j. It is assumed that 
the allocation belongs to some convex region < iS > and that Qj : R+ — >■ R 
is an increasing concave function. 
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Switch networks serve jobs one-by-one subject to scheduling constraints 
in a first-in-first-out manner. By contrast, bandwidth sharing networks 
serve jobs simultaneously subject to rate constraints in a processor-sharing 
manner. Bandwidth sharing networks have been shown to be throughput 
optimal in a number of different settings, [2Ul [21 El US] ■ 

So, generalizations of the MaxWeight policy alter the dependence on 
queue sizes, Qj, with an appropriately chosen function, fj. By contrast, 
bandwidth sharing networks alter the dependence on schedules, Sj, with 
an appropriately chosen function, Qj. It would seem possible that one 
could form scheduling policies for switch networks based on network util- 
ity optimization, as described above. To the knowledge of this author, 
discussions to this end are first made by Wischik and Shah [34] and Zhong 
[45| for the weighted a-fair bandwidth allocation policies [23| . 

Melx Weight- (a, g) 

We will discuss precise terms shortly, but, in brief, the MaxWeight-(a, g) 
policy chooses a schedule which optimizes 



where £ Z is the queue sizes for each queue j £ J and where Sj is the 
mean number of jobs served under random schedule in <5>. Here, for 
j £ k7, Qj is a convex increasing function and a is a positive real number. 

The MaxWeight-(a, g) policy generalizes the MaxWeight-Q policy, it is 
myopic, and it incorporates a functional dependence similar to the network 
utility optimization policies, described above. As we will discuss, they 
allow the scheduling analogue of weight a-fair bandwidth allocation |23) . 
Although it is clear we can define a MaxWeight-(/, g) schedule where we 
replace the function with a function fj{Qj), we do not provide an 
analysis of this policy. 

Our stability analysis for these policies can be further justified because 
it is conjectured that such policies can have optimal behaviour. In par- 
ticular, Zhong |45l pg. 102] conjectures the proportional fair scheduling 
policy - where a = 1 and gj{s) — log(s) - has optimal queue size scaling 
in heavy traffic for an input-queued switch. Here scaling is taken both 
with respect to traffic intensity and network size. 

In this paper, we consider the MaxWeight-(Qf, g) as a generalized policy 
for switch networks. So why not consider it as an appropriate general- 
ization for bandwidth networks? This is because the optimal network 
utility maximization ([1]) is not a scheduling decision but an equilibrium 
reached by competing Internet flows. Thus one must flrst argue that the 
MaxWeight-(Q:, <;) objective is achieved by a congestion control protocol 
before such an analysis. 



maximize 




over 



s e<s>, 
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Contribution 

The main contribution of this paper is to prove that the MaxWeight-(a, g) 
pohcy is throughput optimal under independent identically distributed 
arrivals. 

For MaxWeight, the throughput optimality is the foremost property 
associated with this myopic policy. So, although there are a raft of other 
open problems which one could ask, in this article we focus on addressing 
this first order stability question. 

In the conclusions, we will discuss a number of other open questions 
associated with the MaxWeight-(Qf, (;) policy some of which have been 
answered for the Ma^c Weight policy and for Bandwidth Sharing Networks. 

2 Related work 

As discussed, the MaxWeight policy was first defined by Tassiulas and 
Ephremides [41] and then considered in the context of input queued 
switches by McKeown et al. [21j . The generalized MaxWeight policies 
are subsequently analyzed by a number of authors [T] 1361 [3 [52] . The be- 
haviour of MaxWeight has been studied in a number of limiting regimes: 
fluid limits 81 , heavy traffic ^6] , large deviations [371 138| , and overload 
[34| . Notably, Shah and Wischik 'ST in their study of overloaded networks 
analyse both bandwidth sharing networks and switch networks together 
and, to the best of knowledge of this author, they are this first to sug- 
gest using a network utility function for switch scheduling. Further work 
combining switched networks with bandwidth sharing can be found in 
Moallemi and Shah [H], Zhong [35], and Shah et al. [55] . 

It has been argued that congestion control protocols implicitly solve 
a network utility optimization [151 135] . This led to bandwidth sharing 
networks [2D] , which model the stochastic arrival and departure of utility 
optimizing Internet flows. Analogous to MaxWeight, there are numerous 
stability proofs [9| [3] [44| [lT| [l7| [26] , and these policies have similarly been 
analyzed in different limiting regimes: fluid limits [16] . heavy traffic [14j . 
large deviations [T^], and overload 

We prove the stability of the MaxWeight-(Q, g) policy using method of 
fluid limits adopted by Rybko and Stolyar ^S], Dai [S1[S], and Bramson 
[3]. This method is applied to the MaxWeight policy by [S]. However, 
our proof more closely follows the fluid analysis of Kelly and Williams 
[16| and stability proof of Bonald and Massoulie [5| which are applied to 
bandwidth sharing networks. 

MaxWeight policies apply to switched networks and authors have in- 
vestigated numerous applications. These include wireless networks |41) . 
input queued switches [21], data centers [34|, semiconductor wafer fab- 
rication [7], road traffic management [43], on-line auctions [39] and call 
centers [IB] . 
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3 Structure 



In the next section, we introduce the main notation used throughout the 
paper. In Section [S] we formally define the MaxWeight-(Qf, t;) policy and 
we state the main result of the paper: Theorem [T] We prove Theorem [1] 
by analyzing the stability of an appropriate fluid model. We define this 
fluid model in Section [S] We, also, state Proposition [T] This fluid limit 
result justifies the fluid model's connection with the MaxWeight-(Q, g) 
queueing process. The proof of Proposition [T] is given in the Appendix. 
In Section [T] we prove the stability of our fluid model. This proof is the 
crux of the paper. Following this, in Section [S] we combine Propositions 
[T]and[2]to prove the main result, Theorem[T] We then conclude the paper 
discussing possible lines of further research. 

4 Queueing Network Notation 

We define a discrete-time queueing network where there are restrictions 
on which queues can be served simultaneously. 

We let the finite set J index the set of queues. We let a(t) — {aj{t) : 
j £ J) £ Z:^ be the number of arrivals occurring at each queue at time 
i G N. We assume {a{t)}'^i is a sequence of independent identically dis- 
tributed random vectors with finite mean a € (0, oo)"^ and finite variance 
E[aj{tf] < K for all j e 

We let the finite set S be the set of schedules. Each g — {aj : j £ 
e7) G <S is a vector in Z'^ where aj gives the number of jobs that will 
be served from queue j under schedule a. We assume the zero vector, 0, 
belongs to 5. We let < 5 > be the convex combination of schedules in 
5. We assume <5> has a non-empty interior. Note the extreme points 
of < 5 > are a subset of S. So any point in < <S > can be expressed as a 
convex combination of points in <S. 

For g, s G R, we define q A s — min{g, s}. If q, s G R"^ then q A s = 
{qj t\ Sj : j & J). For a vector Q G Z^, we define 

5 A Q = {cr A Q : (7 G 5}. 

We let < 5 A Q > be the convex combination of points in 5 A Q. Notice, 
to prevent a queue having a negative number of jobs, a A Q is the number 
of jobs that would be served under schedule a when Q gives the vector of 
queue sizes. 

We let Q(0) = (Qi(0) '■ j & J) be the number of jobs in each queue 
at time t = 0. From a sequence of schedules {<j{t)}'^i, we can define the 
queue size vector Q{t) = {Qj(t) : j £ J) by 

Q,{t + I) = Qj{t) - <7,(t + 1) + a,{t + I), (2) 

for j G J , and f G N. Recall, for positive queue sizes, it is required that 

crj(f + I) G<SAQ(t), 

^The assumption of finite variance is not essential; however, it allows bounds more conve- 
nient for our proofs. 
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for all t € Z+0 

Given {Q{t) : t G N) defines a Markov chain, we say the queue size pro- 
cess is throughput optimal if it is positive recurrent whenever the vector 
of arrival rates, a, belongs to the interior of <<S>. 



5 Max Weight- (a, ^) 



We now define the MaxWeight-(a, g) policy. We let a be a positive real 
number and, for each j £ J , we let gj : R+ ^ R be a strictly increasing, 
differentiable, strictly concave function. Given a queue size vector Q{t — 
1) = iQj{t -1) ■■ j € J) at time t - 1, we define a{t) = {aj{t) : j € J) to 
be a solution to the optimization 



maxumze 



J2gj{s,)Qj{t~ir (3a) 

over s £<S AQ{t-l)> . (3b) 

In general, a{t) need not belong to the set of schedules S A Q{t — 1). 
However, a{t) is a convex combination of points in < 5 AQ{t — f ) >. Thus 
we let a{t) be a random variable with support on iS A Q{t — 1) and mean 
(T(t). The MaxWeight-(Q, jr) scheduling policy is the policy that chooses 
schedule a{t) at each time t £ N. 

To be concrete, we assume the random variables a{t) are, respectively, 
a function of Q{t—1) and an independent (uniform) random variable. Ex- 
pressed differently, E [a{t)\Q{t — 1)], t = 1,2, are independent random 
variables. This ensures that the queue size process {Q(i)}t^Lo associated 
with the MaxWeight-(a, (?) scheduling policy is a discrete-time Markov 
chain. Further, the constraints l|3bp ensure that a schedule never exceeds 
the queue it serves. This said, the constraint induced by queue size vector 
Q{t — 1) is only relevant when a queue may empty and, indeed, could be 
chosen in other ways for these instances. 

For later use, for q G R;^, we let a{q) be the solution to the optimiza- 
tion problem 

maximize 9j{^j)<lj' (4a) 

over s e<S A g> . (4b) 

Remark 1. We remark that the MaxWeight-{a, g) policy coincides with 
the MaxWeight-a policy when gj{sj) = Sj and with the MaxWeight policy 
for a = 1 when gj{sj) = Sj . An addition class of further interest is the 
a-fair policies which, for a > 0, is defined from the maximization 



max 

se<SAQ(t-i)> 



max \os{sAQAt — 1), if a = 1. 

■AQ(t-l)> ^ ^ Jl-^J\ 



se<SAQ{t-i)> 



■^Observe, this choice of notation is equivalent to defining the queueing process by Qj{t ■ 
1) = [Q^it) - cTjit + 1)]+ + aj{t + 1). 
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As discussed in the introduction, the a-fair policies were first introduced 
and studied in the context of Bandwidth Sharing Networks, \23[ For 
switch-networks, the a-fatr policy is first defined by Shah and Wischik 134^ - 
The a = 1 policy is referred to as the proportionally fair policy 'Wl. For 
input-queued switches, it is conjectured that the proportionally fair switch 
scheduling policy has optimal queue size scaling in heavy traffic 145^ - 

The main result of this article is the following proof of throughput 
optimality. 

Theorem 1. For the MaxWeight-(a, g) policy, if the vector of average 
arrival rates a belongs to the interior of <S> then the queue size process 
{Q{t),t £ Z+) is positive recurrent. 



6 Fluid model 

In this section, we state a fluid model and a fluid limit associated with the 
MaxWeight-(Q:, g) policy. The proof of the fluid limit is given in Appendix 

in 

A positive, absolutely continuous process q{t) = {qj{t) : j £ J), t £ 
R+, is a fluid model for the MaxWeight-(Q, g) policy if, for j £ J and for 
almost ever jjfl time t £ R+ , 

^=a,-a*(g(t)), if*W>0, (5a) 

^=0, if*(t)=0, (5b) 

where o-*(g) solves the optimization 

maximize ^^3i('Sj)gj* (6a) 

over s £<S> . (6b) 

The key distinction between o*(ci), above, and (T(g), defined by optimiza- 
tion (Q, is that we remove the constraint where we cannot schedule more 
jobs than there are in each queue, i.e. compare (|4b|l with (|6b[l . 

We can formalize the sense that g(t) is the limit of the MaxWeight- 
(a,g) policy. We let {Q'''^^}ceN be a sequence of versions of our queueing 
process for the Max Weight- (a, g), where ||(5(0)| |i = c. Here and hereafter, 
II • 111 is the I/i-norm. We define 

(7) 

for t £ R-|_. The following result formalizes a fluid model q, ((S)), as the 
limit of {Q'^^'jcgN. In informal terms, it states that the only possible limit 
of the sequence {Q'^^'jceN as c — ^ oo is a process q satisfying ((5]). 



^By almost every, we mean on all points except for a set of Lebesgue measure zero. 
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Proposition 1 (Fluid Limit). The sequence of stochastic processes {Q'^^'Icen 
is tigh^ with respect to the topology of uniform convergence on compact 
time intervals. Moreover, any weakly convergent subsequence o/{Q'"'^'}ceN 
converges to a Lipschitz continuous process satisfying fluid equations ^ . 

The proof -and indeed statement- of Proposition [T] is somewhat technical. 
However, the main point is that we can compare the majc-weight queueing 
process to a tractable fluid model, q. Proposition [T] is proven in Appendix 
El We now analyse the stability of the fluid model q. 

7 Proof of Fluid Stability 

We now consider a process q{t), t G R+, that satisfles the fluid limit 
equations ((SJ. In the following theorem, we show that these fluid solutions 
are stable in the sense that they hit the zero state in finite time. This 
result will be sufficient to prove positive recurrence of the MaxWeight- 
(a, g) queue size process. 

Proposition 2 (Fluid Stability). There exists a time T > such that, 
for every fluid model {q{t) : t G R+) satisfying (O and with ||g(0)||i — 1, 

%(t) = 0, jGj, (8) 

for all t > T. 

The main idea is to consider the gradient of the tangent line of the 
MaxWeight-(a, g) objective, ({Gap , between two points: the arrival rate 
and the optimal solution, see (|10p below. Integrating this obtains a Lya- 
punov function (|12|) . This idea is used by Bonald and Massoulie [3] in 
their analysis of weighted a-fair bandwidth sharing networks. For switch 
networks. Proposition [2] follows analogously. 

Proof. We define Gq{s) to be the objective of the MaxWeight-(Qf, <;) opti- 
mization, 

G,{s) = J29AsM- (9) 
Recall that in our fiuid equations 

a*{q{t)) e argmax Gq(t)(s). 
se<s> 

Any vector p belonging to the interior of < 5 > is not optimal. Thus 
Gq{t){p) < G,(t)(o-*(g(f))). As Gq(t){-) is strictly concave, Gq(t){-) must 
be increasing along the line connecting p to cr*(g(f)). In other words, for 
aU pe<S>° and for q{t) / 0, 

(a*(g(f))-p)-VG,(t)(p)>0. (10) 

Recall a sequence of random processes {Q(=)}c is tight if every subsequence of {Q'^^'jc 
has a weakly convergent subsequence. 
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Here VGq(p) — {g'jipj)qj{'t)" ■ j G JJ)- Since a belongs to the interior of 
<5>, there exists e > such that {l + e)a £<S>. We define p — {l + e)a. 
In this case, we can re-express the inequahty (|f Up as follows 

" f^j*('3'(*)))ffj(Pi)'?iW" ^ -e^] f^PjOftC*)"- (11) 
As ^ = (ij — a*{q{t)), we define the Lyapunov function 

q G R:J. The function is positive and L{q) = iff =0 for all 

j (z J ■ We now observe 

^^MH = E (a, - a*Mt)))g',{pMAtr 

<^^J29'Apj)<iAtr- (13) 

The equality holds by the chain rule and the inequality holds by (|f f p . 
We define norms on M.'^ 

||q||l+. = (L(q))TT^, (f4) 

Mc. = {J29'Ap^)iAtr)'- (15) 

By the Lipschitz equivalence of norms, there is a constant 7 > such that 

7!klli+« < IklU, 

for all q G R:jlf| Applying this observation to the inequality (|f 3p . we see 
that 

'J^<-n-Liqit))Tf.. (16) 

Observe, by the above inequality, if L{q{T)) = for any differentiable 
point T then L{q{t)) = for all t > T. Now, whilst L(q{t)) > 0, we have 
from (IT6l) that 



i(g(t))T+^ -L(g(0))T+^ 
= ^\l + a)-L(,W)^^^M))rfi 

< -e(l + a)-S°'t 
Rearranging this expression, we see that for all times t 

mt)) < (Liqm^h _ e(l + a)-S"t)'^" . (17) 



^ Note, < (1 + 0^) I'^l maxj g^. (pj )qj and also note that maxj g'j(pj)qj < \\q\\ 

So, for instance, we can take 7 = (1 + a) 1+° |J7'|~"'". 



9 



The function L{q) is continuous and therefore bounded above by a con- 
stant, K, for all values of q with — 1. Hence, if ||g(0)|ji = 1, 
L{q{t)) = for alH > T where 

and thus, as required, qj{t) = 0, j £ J , for all t >T. □ 



8 Proof of Positive Recurrence 

We are now in a position to combine Propositions [T] and [5] to prove The- 
orem [T] We could at this point apply the general stability results of Dai 
O |6] and Bramson [4]. However, for completeness we provide a self- 
contained proof. 

Proof of Theorem Ql For every t > 0, the sequence queue sizes {Q''^' (i) }c6N 
is uniformly integrable. This is proven in Lemma [2] in Appendix IB] By 
Proposition [T] for any unbounded sequence in N, there is a subsequence 
{cfcjfcgN for which Q^'^''^ converges in distribution to fluid solution q. Let 
T be the time given in Proposition (2] where qj{T) — for j G J7. Since 
{|Q''^''^(T)|}c^ is uniformly integrable and converges in distribution to 
q(T), we also have Li convergence 

lim E|!Q<^'''(T)||i =E||g(r)||i = 0. (18) 

This implies there exists a k. such that for all c > hi 

E||Q(^'(T)||i < (1-6). (19) 

Note that if H19I) did not hold then we could find a subsequence for which 
(|18|) did not hold; thus, we would have a contradiction. Expanding this 
inequality (|19p . we have as described by Bramson [4], the following mul- 
tiplicative Foster's condition: for ||(5(0)||j^ > k, 

E [||Q(r||Q(0)||i)||^ - ||Q(0)||^|q(0)] < -e||Q(0)||^. (20) 

We now use this to prove positive recurrence of the event { 1 1 Q(i) 1 1 1 — '^'} • 
We consider our Markov chain, Q, at specific stopping times: tq := and 
for n G N, 

Tn ~ r„_i + r||(5(r,i_i)||i, if ||Q('r„-i)||i > K, 

r„ := r„_i -f 1, if ||Q(Tn-i)j|i < k. 
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We define the stopping time A'^ = min{n : r„ > t}. Now, 
<E\\Q{tn)\\i 



<E 



=E||Q(0)||i +E J2 IIQ(^'Olli - 

.n=l 

JV 

Q(0)l|i -6e[^ l|Q(r„_i)||il[l|Q(r„_Olli > ^]] 

n = l 
JV 

+ ||a||iE[^l[l|Q(r„-i)||i <fc]] 
?i=i 

JV 

E||O(0)||i- jE[^(r„- 



(21) 



l)l[llO(^n-l)||l > k]\ 



JV 

+ ||a||iE[;^l[||Q(r„_i)||i <«]] 



=E||O(0)||i --E[r^] 



+ 



<E||O(0)||i--t 



+ Ml + 



z 

J)e[^I[||Q(s)||i<ac]] 



(22) 



(23) 



(24) 



In inequality (|21[) . we condition on the event {||Q(T"n-i)||i > k} and apply 
the multiplicative Foster's condition (|2Up : for equality (|22|l . we observe 
that, by definition, r„ — r„_i = r||Q(r„_i)||i and then adding appropriate 
terms we get an interpolating sum for equation ((23]); and finally, for ((24}, 
we observe the sequence ||Q(T"n)||i, for r„ < t, must have hit below k less 
times than the sequence ||(5(s)||i, s < t. 

Rearranging the positive expression (|24[) . dividing by t and taking 
limits, we gain 



1 * 
liminf-E[^l[||Q( 



s)lll > 



T\\a\\i+e 



> 0. 



Thus, Q is positive recurrent because it is positive recurrent in the finite 
set of states {q £ TLJ^ : ||g||i < k}. □ 



9 Conclusion 

We have now shown that the MaxWeight-(a, g) policy is throughput op- 
timal. There are many other questions which could be asked for these 
policies. 

The first of which was the conjecture of Zhong [45j on the queue size 
scaling of the proportionally fair scheduling policy. A proof of this opti- 
mal behaviour further necessitates extensions beyond Max Weight. More 
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immediately, one could prove throughput optimality of MaxWeight-(/, g), 
in a similar manner to that considered by Meyn [21]. The analogous Back- 
Pressure policy could be considered in a similar manner to |34| . Given 
that arrival rates may vary over time, one could further prove universal 
stability as considered by Neely [23- A number of regimes beyond the 
fluid limit in this paper could be considered: heavy trafflc [55] i overload 
[34) . large deviations [371 138) . Further, we do not analyse the computa- 
tional complexity resolving scheduling solutions. Efficient solution of the 
MaxWeight-(Q;, g) optimization is of course important to any practical im- 
plementation [401130) . We do not consider the effect of changes in network 
topology on stability [H]. Further, we do not consider decentralized im- 
plementation of this policy which has been a topic of recent investigation 
[291 [13]. 

Although there are many avenues that may be further pursued, we 
have first affirmatively answers this fundamental stability question for 
the MaxWeight-(a, g) polices. 

A Fluid Limit 

Proof of Proposition [7] To prove the tightness of a sequence of process 
{Xt'^'jc, from Robert TP. Theorem C.9], we see that we must prove 

limpf sup Wx^^^v) ~ X^''\u)\\ >r]]=0. (25) 

S^O \ u,v:u,v<t II / 
^ \v-u\<S ' 

We wish to demonstrate this for {Q'^^'jc. For j G J , we can express 
Q^f (t) as follows 

Qf (t) = Qf (0) + % ^ - 5f ' (t) + A?j^' it) + TVj^' (t) (26) 
where, from a{t),a,a{t),a{t), we define 

s = l 
[ctj/c 

s = l 

for j G J. 

It is clear Qj{0) + aj [ctJ/c satisfies ([25]). By the triangle inequality, it 
is sufficient to show ()25p holds for each of the above terms. 

The term a{s) is bounded by some constant Ki for all s. So, we have 
the Lipschitz condition 

\Sj{t) - Sj{s)\ < Ki\t ^ s\ + 2Kic-^ 
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which satisfies (|25p with 5 < r]K^ ^ . 

The process Mj {t) is a martingale thus by Doob's L2 inequahty 

sup I («) - Mj"' (w) j > 7? I (27) 

\v — u\<i5 

<P ( sup \m\''\u)\ > ^ ] (28) 



-E(a,(l)-a,)= 



0. (29) 



The process Nj'^'' (t) is a Martingale with summands bounded by some 
constant K2/C, so by the Azuma-Hoeffding Inequality 

P(|iVj'='(f)j > »?) < 2e~^ (30) 
Again applying Doob's L2 inequality 

sup I iVj"' (v) - iVj"' (ii) I > r? ) (31) 

<pf sup jiVf (u)| > 2") (32) 

<±E[iVf (t)^] < .0. (33) 



Thus our sequence of processes {Q^^^'jc are tight. 

Note the inequalities above, (|27p and pif) . also prove that the se- 
quences iVt'^' and Mt'^' converge in distribution to zero. 

It remains to show that any fluid limit process satisfies the fluid equa- 
tions ([5]). By the Skorohod Representation Theorem |27l Theorem C.8]] 
and since {Q^^^c is tight: for any convergent subsequence of we can choose 
a subsequence where convergence occurs almost surely, along an appro- 
priately chosen probability space. Thus, using this and the definition of 



lim Qf'(t) (34) 

- lim |q("' (0) + a, - (t) + Mf (t) + (t) } (35) 
:(7j(0) + a^t - lim S'f\t) (36) 



/[cij /c 
Gj{Qf{cs))ds = qj{t). (37) 



As G* is bounded it is clear that qj(t) is Lipschitz continuous. If qj{t) > 
then, by continuity, qj(s) > for an open region s G {t — S,t -\- 5). We 
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have for this choice of t and s 

(38) 
(39) 

(40) 

(41) 

As Lemma [1] states d-j{qj) is continuous for qj > 0, we apply the bounded 
convergence theorem in the second equality above. Now, in addition to 
being tight, we have proven the fluid equations, (O, hold. 

□ 









■aj {t - 


C 


lim / aj{Q\'^' {cu)du 


-Uj it - 




/ lim (Tj{Q^^' {cu)du 

[csj/c 


:aj{t - 




1 G*{qj{u))du 

[csJ/c 



B Other Lemmas 

The following lemma assists our fluid limit proof. Proposition!!] Recall our 
definitions of (j{q), (U), and cy*{q), We let {g'-'^'jc be any unbounded 
sequence in with = c. From this, we define g''^' = q^'^^ jc. 

Lemma 1. // g''^' q as c oo then, for each j & J with qj > 0, 



Proof. We can take an appropriate subsequence and without loss of gen- 
erality, we may assume that the bounded sequence {o"(?'''^')}c converges 
to some value a. In addition, we take M such that M > Oj for all cr G <S 
and j £ J and define k £ R^^ by 

M, if qj > 0, 
0, if qj = 0. 



Notice, since g''^' — q 

eventually as c — !> cxd. 
By definition 



g'"' > n (42) 



E*('^^)9"<E*K(9))9"- (43) 

jej jej 

By strict concavity: g^ (psj + (1 -p)sj)gj > pgj{sj)qj + {l-p)gj{s'^j)qj, 
the optimal solution o-*(q) is unique for each component j with g^ > 0. 
We define G* by 

G* = ma« E diisMl (44) 
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By definition CT*{q) is optimal for the above optimization. Also, notice 
the above optimum remains equal to G* if we maximize over < <S A k > 
instead of <5>. Thus 

= lim max ^ ffj (crj)(gj''')" (46) 



< lim max ^ (a,) (gj^^)" (47) 

-i™ E^'^■(^^■(^*^'))(#T (48) 

The first equality, above, holds optimum remains equal to G* if we max- 
imize over < S A K > instead of < S >; the second equality holds by 
continuity of the objective function and compactness of < <S A k >; the 
next inequality holds by monotonicity (I42p : and the remaining equalities 
hold by definition and continuity. 

Thus a is optimal and thus aj — o-j{q) for all j with qj > 0. So, as 
this limit aj is unique for all subsequences the lemma must hold. □ 

Lemma 2. For every t, the random variables {Q''^' (t)}cgN are uniformly 
integrable. 

Proof. It is sufficient to prove that these random variables are bounded 
above by a sequence of L2-bounded random variables. The queue size is 
bounded above by the number of arrivals, so, for all t > 

jej jej s=i 

By assumption, aj{s) has finite variance; thus, the right-hand side of 
inequality (|50p is L2-bounded; and so, X^^gj- Qj"^' (i) is uniformly inte- 
grable. □ 
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